-
Notifications
You must be signed in to change notification settings - Fork 114
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
✏️ Rewrite KVStore, ClientStats, and CommHelper #1040
✏️ Rewrite KVStore, ClientStats, and CommHelper #1040
Conversation
storage.ts is replacing storage.js which had the KVStore service inside it. storage.ts will provide a set of functions performing the same duties as KVStorage's functions did. - `get` -> `storageGet` - `set` -> `storageSet` - `remove` -> `storageRemove` - `getDirect` -> `storageGetDirect` - `syncAllWebAndNativeValues` -> `storageSyncLocalAndNative` storage.ts will still use the "BEMUserCache" Cordova plugin in exactly the same way that KVStore did. However, instead of using the `angular-local-storage` package as a wrapper around localStorage, we will use it directly. A couple functions were added ('localStorageSet` and `localStorageGet`) to facilitate using localStorage directly - localStorage requires us to stringify objects before storing, and parse them on retrieval. Other than these substitutions, and being rewritten in modern JS with some typings, the logic is exactly the same as it was in KVStore. --To facilitate this change, storage.js is temporarily renamed to ngStorage.js so that it doesn't conflict with the storage.ts filename. ngStorage.js will be removed soon after it is not used anymore.
storage.ts replaces KVStore, so we can replace all uses of KVStore throughout the codebase, substituting the new functions from storage.ts as follows: - `get` -> `storageGet` - `set` -> `storageSet` - `remove` -> `storageRemove` - `getDirect` -> `storageGetDirect` - `syncAllWebAndNativeValues` -> `storageSyncLocalAndNative`
Rewritten into storage.ts, this Angular service is not needed anymore. We can remove the file, remove it from index.js, and remove it as a dependency from all of the Angular modules that previously used it.
Followed the advice of https://stackoverflow.com/a/74309873 so that we can have a mocked out localStorage to use in tests
To test storage.ts we need to be able to mock the usercache cordova plugin, as well as fill in the missing window.Logger. I created mocks for both of these which are kept in the __mocks__ folder and simply called at the start of storage.test.ts. Because storage.ts calls getAngularService, angular-react-helper.tsx was included in the process. That file imported PaperProvider, which did not work with jest. Because we are not angularizing components anymore, we do not actually need much of the code in angular-react-helper anyway. Let's just take it out so we can safely implicate getAngularService. As for the storage tests themselves, they check whether the storage functions are able to store and retrieve data, and crucially they ensure that local and native storage can compensate for each other if one of them gets cleared.
js/plugin/clientStats.ts will replace js/stats/clientstats.js - camelcase filename convention - we don't need a dedicated folder for stats because clientstats.js was the only thing in it Nothing significant has changed here. I just refactored into modern JS and added type definitions for each function's parameters. mappings of old/new exported methods: - getStatKeys -> statKeys (now a variable, not a function) - getAppVersion -> getAppVersion - getStatsEvent -> getStatsEvent - addReading -> addStatReading - addEvent -> addStatEvent - addError -> addStatError
ClientStats was rewritten into clientStats.ts. This commit simply substitutes in the new methods for the old ones, everywhere that client stats are recorded, as follows: - getStatKeys -> statKeys (now a variable, not a function) - getAppVersion -> getAppVersion - getStatsEvent -> getStatsEvent - addReading -> addStatReading - addEvent -> addStatEvent
Rewritten into a new file clientStats.ts, this old file is not needed anymore. We can remove the file, remove it from index.js, and remove it as a dependency from all of the Angular modules that previously used it.
This is used in ProfileSettings is for the "App Version" row at the very bottom of the Profile page. getAppVersion() was not working correctly - the correct function to call is `getVersionNumber` which returns a promise. In a .then block, we can memoize this value in the local let appVersion. Since it is memoized, we don't need a dedicated state value in ProfileSettings for appVersion. We can just call getAppVersion() directly where it's used in the SettingRow.
There are a few other 'window' variables that are provided in a Cordova/Ionic app and are expected in various few places throughout the codebase. To test, we need to be able to mock these too. mockCordova and mockDevice just provide fake platform/version info. mockGetAppVersion mocks the cordova-plugin-app-version, which is a third-party plugin we use to get the version of the app. This is used in clientStats.ts, so we will need to use this mock when we test that. The mock returns info for a "Mock App", version 1.2.3.
getAppVersion() calls cordova-plugin-app-version's getVersionNumber, which is asynchronous. So getAppVersion should be asynchronous too. I changed it to return a promise. We handle this in clientStats by adopting async/await syntax. We handle this in ProfileSettings by storing it as a ref once the promise is resolved. Then we just access it as appVersion.current.
Tests the methods exported from clientStats.ts, including getAppVersion, addStatReading, addStatEvent, and addStatError. The tests record these stats and then query the usercache for the messages, expecting to see a new entry filled in with the fake data. putMessage and getAllMessages needed to be added to the mockBEMUserCache, since the client stats are recorded as 'messages' rather than 'key-value' pairs
CommHelper from js/services.js is being rewritten into commHelper.ts. This commit moves over the functions from CommHelper that we need. So now the following functions are exported: - fetchUrlCached (was already in commHelper.ts) - getRawEntries - getPipelineRangeTs - getPipelineCompleteTs - getMetrics - getAggregateData - registerUser - updateUser - getUser The following functions were not moved over to the new commHelper.ts because they are not currently used in the codebase and I don't anticipate them being used in the foreseeable future: - putOne - getTimelineForDay - habiticaRegister - habiticaProxy - moment2Localdate - moment2Timestamp - getRawEntriesForLocalDate
CommHelper from js/services.js was rewritten into js/commHelper.ts. This commit switches all functions calls to the new commHelper.ts. All of the functions are named the same and perform the same logic.
Rewritten into commHelper.ts, this Angular service is not needed anymore. We can remove the 'factory' declaration from js/services.js and remove it as a dependency from all of the Angular modules that previously used it.
I rewrote TODO: write tests for |
…JGreenlee/e-mission-phone into rewrite-services-sept2023
-Added one test for `fetchUrlCached` - uses a mocked 'fetch' to simulate retreieving data from a URL. It ensures that the second time this URL is queried, the data comes back faster (because it gets cached in the localStorage after the first call) Note at the bottom explains why other functions were not tested.
All of the functions in So to test these at all, we would have to mock the entire For now I am going to test the one function I can (even this requires mocking the @shankari Is this acceptable to you? |
@JGreenlee ah I finally see what you were referring to at the team meeting. I agree that unit testing However, fully testing LMK if that is clear. |
The approach is clear. Although, I don't understand how the integration testing would be implemented. To actually make a call to the server, we would need to make native calls to the BEM server comm plugin, which then would query the server. How would we interface with Cordova plugins through a Github workflow? And as a more general question, why do we make server calls through native code anyway? Couldn't we do it more easily in JS? |
Quick reply to document what we discussed in the meeting:
since this involves native code, @louisg1337 might be interested/want to help as well. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very clean PR, most changes can be deferred until later.
const mockBEMUserCache = { | ||
getLocalStorage: (key: string, isSecure: boolean) => { | ||
return new Promise((rs, rj) => | ||
setTimeout(() => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume that the setTimeout
is to delay the response.
future fix: you could pass in the timeout as a parameter to the mock (assuming that something like mockBEMUserCache(delay)
works and then we could test what happens if the plugins are slow.
return unmungeValue(key, localStorageGet(key)); | ||
} | ||
|
||
function findMissing(fromKeys, toKeys) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
future fix: It would be good to add a unit test for findMissing
as well - it is covered in storageSyncLocalAndNative
, but sometimes you can test more extensively by calling the lower function directly (e.g. call it with a blank list, with null etc)
per https://github.com/e-mission/e-mission-phone/pull/1040/files#r1351511049, removes the backwards compat to munge when filling in native values from local storage. Also removes the backwards-compat comment, replacing it with other comments describing how we fill in missing local or native values
A user-facing popup here is likely to just annoy or confuse users. Let's set these log statements at the "WARN" level so it is more visible than other log statements if we do need to debug it, but not intrusive or detrimental to UX
It is fairly self-explanatory that {key: value} was the chosen approach
Per https://github.com/e-mission/e-mission-phone/pull/1040/files#r1351477569, we'll show an error message here if the BEMUserCache 'db' is not defined. We'll also ensure that if the Logger plugin is undefined, we do not try to call it as this would cause an error.
A couple of the functions that were excluded from the rewrite were requested to be added back: e-mission#1040 (comment) The logic is the same, but with modernized syntax, and using Luxon instead of Moment to deal with timestamps
Appending "user-friendy" descriptions to error popups is already handled in logger.ts (see `displayErrorMsg`), so it is unnecessary in this file.
- Add the 2 functions to the list that were re-implemented after the initial rewrite - More accurately describe how we would ideally test server comm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is now done. I am merging to unblock the others...
As part of the ongoing effort in e-mission/e-mission-docs#977, these are a few of the services that I have rewritten upfront.
These services are widely used throughout the codebase. It's preferable to rewrite these first before attempting the majority of the other services because so many of the other services depend on these 3.
This PR also adds tests for the rewritten files. In doing so, I created mocks for a few of the cordova plugins that are used throughout the codebase - this will benefit others in writing their own tests.